Tips and tricks for Julia practitioners — Julia Eindhoven
Patrick Altmeyer
Blurb
As a Julia practitioner you may want to publish your work in various forms: notebooks, Markdown, HTML, PDF and more. What if you could produce all these different outputs from the same input? I will share how I’ve been using Quarto with Julia, for package documentation, blogging and JuliaCon proceedings.
DISCLAIMER: Views presented in this presentation are my own. I am not affiliated with either Quarto or Posit (RStudio).
Quick Intro
Currently 2nd year of PhD in Trustworthy Artificial Intelligence at Delft University of Technology.
Working on Counterfactual Explanations and Probabilistic Machine Learning with applications in Finance.
Previously, educational background in Economics and Finance and two years at the Bank of England.
Enthusiastic about free open-source software, in particular Julia and Quarto.
What is Quarto?
From R Markdown …
R Markdown users have enjoyed many of the benefits highlighted in today’s for many years.
For me personally, the workflow enabled by R Markdown was for many years a key reason to rely on R whenever possible (see here).
In recent years Posit (formerly RStudio) has first embraced Python and then geared towards multi-language support.
We have some wonderful news: RStudio is now Posit! 🎉
While many things will stay the same, our rebrand will result in changes beyond a new name. To start, our new website https://t.co/vI56Gz7Yqf is now live. Please check out our new home and let us know what you think! pic.twitter.com/hzJGXsX0tj
Generate multiple different output formats with ease:
The old school: \(\LaTeX\) and PDF (including Beamer); MS Office
The brave new world: beautiful, dynamic HTML content
websites
e-books
apps
…
All of this starting from the same place …
A plain Markdown document blended with your favorite programming language and a YAML header defining your output.
Effective Communication and Reproducibility in Science
Most science today involves code. Often code forms such an integral part of the science, that it deserves its place in the final publication.
Scientific Ideas can often be most effectively communicated through dynamic visualizations.
Requirements and preferences vary.
Quarto allows us to cater to those needs, while at the same time facilitating reproducibility by bridging the gap between computations and writing.
Quarto enables effective communication and reproducibility without compromises.
Code chunks
Most science today involves code.
usingMarkdownMarkdown.parse("""Often code forms such an integral part of the science, that it deserves its place in the final publication.""")
Often code forms such an integral part of the science, that it deserves its place in the final publication.
Using simple YAML options, we can specify how code is displayed. For example, we may want to use code folding to avoid unnecessary interruptions or hide large code chunks like this one that builds Figure 1.
Code
usingJavis, Animations, Colorswww_path ="www/images"size =600radius_factor =0.33functionground(args...)background("transparent")sethue("white")endfunctionrotate_anim(idx::Number, total::Number) distance_circle =0.875 steps =collect(range(distance_circle,1-distance_circle,length=total))Animation( [0, 1], # must go from 0 to 1 [0, steps[idx]*2π], [sineio()], )endtranslate_anim =Animation( [0, 1], # must go from 0 to 1 [O, Point(size*radius_factor, 0)], [sineio()],)translate_back_anim =Animation( [0, 1], # must go from 0 to 1 [O, Point(-(size*radius_factor), 0)], [sineio()],)julia_colours =Dict(:blue =>"#4063D8",:green =>"#389826",:purple =>"#9558b2",:red =>"#CB3C33")colour_order = [:red, :purple, :green, :blue]n_colours =length(julia_colours)functioncolor_anim(start_colour::String, quarto_col::String="#4b95d0")Animation( [0, 1], # must go from 0 to 1 [Lab(color(start_colour)), Lab(color(quarto_col))], [sineio()], )endvideo =Video(size, size)frame_starts =1:10:40n_total =250n_frames =150Background(1:n_total, ground)# Blob:functionelement(; radius =1)circle(O, radius, :fill) # The 4 is to make the circle not so smallend# Cross:functioncross(color="black";orientation=:horizontal)sethue(color)setline(10)if orientation==:horizontal out =line(Point(-size,0),Point(size,0), :stroke)else out =line(Point(0,-size),Point(0,size), :stroke)endreturn outendfor (i, frame_start) inenumerate(1:10:40)# Julia circles: blob =Object(frame_start:n_total, (args...;radius=1) ->element(;radius=radius))act!(blob, Action(1:Int(round(n_frames*0.25)), change(:radius, 1=>75))) # scale upact!(blob, Action(n_frames:(n_frames+50), change(:radius, 75=>250))) # scale up furtheract!(blob, Action(1:30, translate_anim, translate()))act!(blob, Action(31:120, rotate_anim(i, n_colours), rotate_around(Point(-(size*radius_factor), 0))))act!(blob, Action(121:150, translate_back_anim, translate()))act!(blob, Action(1:150, color_anim(julia_colours[colour_order[i]]), sethue()))# Quarto cross: cross_h =Object((n_frames+50):n_total, (args...) ->cross(;orientation=:horizontal)) cross_v =Object((n_frames+50):n_total, (args...) ->cross(;orientation=:vertical))endrender( video; pathname =joinpath(www_path, "julia_quarto.gif"),)
Figure 1: A simple animation built with Javis.jl.
Dynamic Visualizations
Scientific Ideas can often be most effectively communicated through dynamic visualizations.
usingPlotsusingStatsBasesteps =randn(1)T =100anim =@animatefor t in2:Tappend!(steps, randn(1)) random_walk =cumsum(steps) p1 =plot(random_walk, color=1, label="", title="A Gaussian random walk ...", xlims=(0,T)) acf =autocor(random_walk) p2 =bar(acf, color=1, label="", title="... is non-stationary", xlims=(0,10), ylims=(0,1))plot(p1, p2, size=(800,300))endgif(anim, fps=5)
Meeting Varying Requirements
Quarto has fantastic support for traditional and modern scholarly writing.
The challenge …
Some people still prefer to read paper or work with MS Office. Most scientific journals, for example, still work with PDF and \(\LaTeX\).
Quarto allows us to cater to different requirements, while at the same time facilitating reproducibility by bridging the gap between computations and writing.
The world and the data that describes it is not static 📈. Why should scientific outputs be?
From dynamic inputs …
The code below depends on remote data that is continuously updated:
Code Execution — You can specify YAML options such that changes to your underlying Julia code will trigger your blog post to be rerendered. This essentially allows you to easily test that the code you publish actually runs:
You get some stuff for free, e.g. citation management. Unfortunately, still no support for cross-referencing …
The use of jldoctest is not always straight-forward (see here). Letting docs run through the Quarto engine provides an additional layer quality assurance.
ConformalPrediction.jl is a package for Uncertainty Quantification (UQ) through Conformal Prediction (CP) in Julia. It is designed to work with supervised models trained in MLJ(Blaom et al. 2020). Conformal Prediction is distribution-free, easy-to-understand, easy-to-use and model-agnostic.
JuliaCon Proceedings
Quarto supports \(\LaTeX\) templates/classes, but I’ve found that rticles still has an edge here.
The list of out-of-the-box templates for journal articles is growing.
CounterfactualExplanations.jl is a package for generating Counterfactual Explanations (CE) and Algorithmic Recourse (AR) for black-box algorithms. Both CE and AR are related tools for explainable artificial intelligence (XAI). While the package is written purely in Julia, it can be used to explain machine learning algorithms developed and trained in other popular programming languages like Python and R. See below for short introduction and other resources or dive straight into the docs.
Turning a nine (9) into a four (4).
A sad 🐱 on its counterfactual path to its cool dog friends.
LaplaceRedux.jl is a small package that can be used for effortless Bayesian Deep Learning and Logistic Regression trough Laplace Approximation. It is inspired by this Python library and its companion paper.
Plugin Approximation (left) and Laplace Posterior (right) for simple artificial neural network.
Simulation of changing posteriour predictive distribution. Image by author.
ConformalPrediction.jl is a package for Uncertainty Quantification (UQ) through Conformal Prediction (CP) in Julia. It is designed to work with supervised models trained in MLJ(Blaom et al. 2020). Conformal Prediction is distribution-free, easy-to-understand, easy-to-use and model-agnostic.
Conformal Prediction in action: Prediction sets for two different samples and changing coverage rates. As coverage grows, so does the size of the prediction sets.
More Resources 📚
Read on …
Related blog post (hosted on this website that itself is built with Quarto and involves lots of Julia content).
Julia to Quarto animation. Source: author (heavily borrowing from Javis.jltutorial)
References
Blaom, Anthony D., Franz Kiraly, Thibaut Lienart, Yiannis Simillides, Diego Arenas, and Sebastian J. Vollmer. 2020. “MLJ: A Julia Package for Composable Machine Learning.”Journal of Open Source Software 5 (55): 2704. https://doi.org/10.21105/joss.02704.